Analyzing experimental paradigms under modification on web-based experiment platforms

Experimental paradigms are particular experiments that can be modified along a variety of dimensions to answer questions different than the original inquiry but which have similar content or structure. For example, the original looking-time study is an experimental paradigm that has shaped developmental psychology. Thomas Kuhn proposed that, under normal conditions, experimentation by modification of previous paradigms is how science progresses. Web-based experiment platforms (e.g., psiturk andWallace) are installed with collections of working experimental paradigms that are pre-populated with structures necessary to use the platform, but which can be modified to allow users to generate novel experiments. Because these experiments are implemented in code, we can identify exactly how the code is modified in practice, and begin to directly measure and even test Kuhn’s hypothesis regarding the progress of normal science. I will describe possible methods for achieving this, ideally providing others a paradigm to modify.